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© A packet data communication network employs 
a local switch, router or bridge device functioning to 
transfer packets between segments of a larger net- 
work. When packets enter this device, an address 
translation is performed to generate local source and 
destination addresses which are much shorter than 
the globally-unique addresses contained in the pack- 
et as dictated by the protocol. These local addresses 
are inserted in a header that is added to the packet, 
in addition to any header already contained in the 
packet. This added header travels with the packet 
through the local switch, router or bridge device, but 
then is stripped off before the packet is sent out onto 
another network segment. The added header may 
also contain other information, such as a local name 
for the source and destination segment (link), as well 
as status information that is locally useful, but not 
part of the packet protocol and not necessary for 
transmission with the packet throughout the network. 
Local congestion information, results of address 
translations, and end-of-message information, are ex- 
amples of such status information. 
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BACKGROUND OF THE INVENTION 

This invention relates to packet data commu- 
nication networks, and more particularly to a way of 
handling packets through a hub, switch or router for 
a network, using a special header attached 'to the 
packets. 

Packet data communication networks of type 
using Ethernet, token ring, or FDDI technologies, or 
other network varieties, Hubs are used for switching 
or routing, or for bridges to additional segments of 
the network. A bridge or router in a computer 
interconnect system is shown in U.S. Patent 
5,020,020, assigned to Digital Equipment Corpora- 
tion. It is typical to use address translation in a 
bridge or router of this type, as shown, for exam- 
ple, in U.S. Patent 4,933,937. The need to translate 
addresses is due to the address length. Some 
protocols or system specify a 46-bit source and 
destination address so that a globally unique ad- 
dress is provided. However, for efficient use of 
resources at a local segment of a large network (as 
within a bridge), it is advantageous to use smaller 
address fields instead of 43-bit addresses, for effi- 
ciency in bit-count of messages as well as effi- 
ciency in processing and storage. For this reason, 
while the 48-bit addresses are carried in the packet 
throughout its lifetime, shorter addresses are gen- 
erated for ioca! routing and processing. Thus, a 
translation mechanism is provided to allow switch- 
ing between global and local addresses. 

Various other information may be locally useful 
in a bridge or router that need not be carried by a 
packet throughout its lifetime. For example, the 
network segments (links) may be locally named 
(assigned an identifying number) so that packets 
may be routed by link number instead of or in 
addition to their local destination node address. 
The packets may be assigned a priority or "service 
class" for local use, aside from the priority at- 
tached to the packet by the protocol being imple- 
mented. Status information may be locally useful, 
but not part of the packet protocol and not neces- 
sary for transmission with the packet throughout 
the network. For example, local congestion infor- 
mation, or results of address translations, or end-of- 
message information, are types of such status in- 
formation. 

SUMMARY OF THE INVENTION 

The invention in its broad form resides in a 
packet data communications network as recited in 
claim 1. The invention also resides in a method of 
operating a packet data communications network 
as recited in claim 5. 

In accordance with one embodiment of the 
invention, a packet data communication network 



employs a local switch, router or bridge device 
functioning to transfer packots between segments 
of a larger network. When packets enter this de- 
vice, an address translation is performed to gen- 

5 erate local source and destination addresses which 
are much shorter than the globally-unique address- 
es contained in the packet as dictated by the 
protocol. These local addresses are inserted in a 
header that is added to the packet, in additibn to 

10 any header already contained 1 in the packet. This 
added header travels with the packet through the 
local switch, router or " bridge device, but then is 
stripped off before the packet is sent out onto 
another network segment. The added header may 

rs also contain other information, such as a local 
name for the source and destination segment (link), 
as well as status information that is locally useful, 
but not part of the packet protocol and not neces- 
sary for transmission with the packet throughout 

20 the network.' Local congestion information, results 
of address translations, and end-of-message in- 
formation, are examples of such status information. 

BRIEF DESCRIPTION OF THE DRAWINGS 

25 

A more detailed understanding of the invention 
may be had from the following description of a 
preferred embodiment, given by way of example 
and to be understood in conjunction with the ac- 
30 companying drawing wherein: 

Figure 1 is a diagram in block form of a commu- 
nications network which may use features ac- 
cording to one embodiment or the invention; 
Figure 1a is an electrical diagram in block form 
35 of a controller for the communications network 
of Figure 1 ; 

Figure 2 is a diagram of the controller of Figure 
1 and ia showing processes executed in the 
controller; 

40 Figure 3 is a diagram of a controller of Figure 1 

and la connected in a network; 

Figure 4 is a diagram of frame formats used in 

the network of Figures 1 or 3; 

Figure 5 is a diagram of the fields contained in 
45 an added header in the formats of Figure 3; 

Figure 6 is a diagram of a data structure used 

for a hash table in the system of Figures 1 and 

1a; 

Figure 7 is a diagram of breadth-first binary 
50 trees used in the method of the invention, to 
store translated addresses; and 
Figure 8 is a flow chart of address lookup proce- 
dures used in the embodiment of the invention 
of Figures 1 and 1a. 

55 
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DETAILED DESCRIPTION OF SPECIFIC EMBODI- 
MENT ", 

Referring to Figure 1, a packet data commu- 
nications network which may use the features of 
the invention includes a controller 10 for interface 
between an FDDI link 11 and a crossbar switch 
device 12. The crpssbsir switch deyjce 12 has a 
number of input/output ports 13, and each one of 
these ports 13 may be. connected by another con- 
troller 10 tp, another network segment 11 such as 
an FDDI link, or a token ring or Ethernet bus, for 
example. The crossbar switch 10 ordinarily .makes 
a directs point-to-point interconnect between one 
port 13 and another port 13, so that, the crossbar 
acts as a -bridge or router in the network, linking 
one network segment to another. A station on a link 
1 1 sends a packet onto; its network segment with a 
destination address which is on another, .different 
segment. The, controller 10 for this segment de- 
tects the address as being that of a station on one 
of. the remote segments, and generates local 
switching information to send to the crossbar 12 so 
that the appropriate interconnect can be made to 
send the packet to the proper port 13 and link 11, 
via another controller 10, As set forth in the above- 
mentioned, copending applications, the crossbar 
switch device can function as ; a .flexible intercon- 
nect device to create a ring or bus using the ports 
13, as well as functioning as a poinMo-ppint con- 
nector as is the usual case for crossbar switches. 

Referring to a more detailed view of Figure 1a, 
each port 13 of the crossbar has a data-in path 14 
and a separate data-out path 15. Jhe interface 
between the controller 10 and the FDDI link 1 1 is 
by way of a media access control (MAC> device 
16, functioning to convert the serial light transmis- 
sion on the incoming fiber optic cable 17 to elec- 
trical pulses, to recover the clock, convert the serial 
data on the optic loop to 6-bit parallel symbols, act 
as an elastic buffer to allow rectocking of data 
entering the controller 10, etc. Of course, all of 
these functions are reversed for outgoing data on 
the cable 18. The interface between the controller 
10 and the MAC device 16 is by an incoming 8-bit 
parallel data path 19a (with additional parity and 
control lines) and. an outgoing 8-bit parallel path 
19b. 

The controller 10 contains a processor or state 
machine 20 to execute various processes as will be 
described, and accesses a packet memory 21 via 
an interface 22, as well as a content addressable 
memory (CAM) 23 via. interface 24. The packet 
memory 21 is addressed by a 20-bit address bus, 
and data is transferred by a 56-bit bidirectional 
data bus, included in the interface 22; various con- 
trol lines are also in the interface 22. The CAM 23 
is driven by a 14-bit bus and various control lines 



in the interface 24. The packet memory 21 is a 
RAM which stores a number of queues for incom- 
ing and outgoing data packets, as well as transla- 
tion tables and hash tables as will be described. In 
5 addition, the packet. pnemory stores certain data for 
which addresses are matched in the CAM 23. 

The controller 10 also interfaces with a line 
, n card processor 25 by bus 26. The line card proces- 
sor .25 is used to execute ^ome diagnostic and 

io initialization functions, and does , not operate in rou- 
tine packet transfer.; 

LpgicaHy, the processor 20 in the controller 10 
executes six independent processes. There are two 
for, inbound packet processing, two for outbound 

75 packet, processing, one for interfacing to the exter- 
nal packet, memory, and one for line card proces- 
sor access.. Packets inbound on FDDI line 17 and 
.going through the controller 10 to the crossbar 
switch ,12 are referred to as "inbound." Likewise, 

20 packets going in the direction of crossbar switch 12 
through the controller 10 to the FDDI output line 18 
are referred to as "outbound." By having indepen- 
dent processes which can operate in parallel, the 
controller 1 0 can process inbound and outbound 

25 packets at full speed. Distributed among the pro- 
cesses are control, parameter and status registers 
that are used to define the operational modes and 
to determine the internal state of the controller 10; 
these registers are accessed through the node 

30 processor interface 26. . 

Referring to Figure 2, the inbound receive (IR) 
process 27 executed on the processor 20 receives 
packets from the interface 1 9a to the FDDI ring via 
the media access control device 16. The IR pro- 

35 cess 27 parses and decodes each incoming packet 
from line 1 9a, places the packet data into an inter- 
mediate FIFO 28, and performs the necessary pro- 
cessing to determine if the packet is to be forwar- 
ded to the crossbar 12. For bridged packets, this 

40 decision is made by performing destination ad- 
dress, source address, and protocol field lookups, 
with the aid of the content addressable memory 23 
and the internal hash function, according to the 
invention. The result of the lookup will determine 

45 where the packet is to go. The packet may have to 
be. transmitted to another port 13 on the crossbar 
12 to reach its destination, the packet may be 
destined for the line card processor 25, the packet 
may be destined for a processing engine 38 in the 

so crossbar switch 12, or the packet may be filtered 
and discarded. When the IR process 27 decides a 
packet is to be forwarded to the crossbar 12, it 
uses the lockup results to generate the information 
in an added header as will be described. It will 

55 select an inbound queue to store the packet in the 
external packet memory 21 and will initiate mem- 
ory requests to transfer the data from its intermedi- 
ate FIFO 28 to the selected queue in the external 
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packet memory 21. The processor 20 executing 
this IR process 27 performs this operation by gen- 
erating IR requests that are sent to the packet 
memory 21. 

The IR process 27 also handles a number of 
exception conditions. The forwarding o1 packets to 
processing engines in the switch 12 can be rate 
limited by software to prevent the traffic from a 
single port from overwhelming the shared process- 
ing resources of the switch 12. When these rate 
limits are exceeded the IR process 27 will discard 
these packets until the rate limit is replenished. 

The inbound transmit (IT) process 29, again 
referring to Figure 2, services the queues in packet 
memory 21 that contain packets to be transmitted 
to the crossbar switch 12. These packets are 
stored in queues in the external packet memory 21 
of the controller 10. When an enabled queue has a 
packet count greater than one, the II process 29 
will begin processing of the packet by initiating 
packet memory requests to move data from the 
packet memory 21 to an internal FIFO 30. Part of 
the data transferred is the added header associated 
with the packet. The IT process 29 uses this added 
header field to request a connection on the cross- 
bar 12 to the desired destination port 13; it re- 
quests this connection by sending information to 
the port interface 13 that indicates the destination 
port of the switch 1 2 and service class information 
describing how the connection is to be serviced. 
Prior to requesting the connection, the IT process 
29 checks the timestamp stored with the packet 
and will discard the packet if it has expired. If the 
IT process 29 determines that a packet has expired 
after a connection has been requested but before 
the connection is made, it will discard the packet, 
abort the requested connection and continue ser- 
vicing the enabled queues. 

When a connection is established, the IT pro- 
cess 29 transfers the data from its intermediate 
FIFO 30 to the crossbar 12 via path 14 and per- 
forms the proper packet formatting during transmis- 
sion. 

The outbound receive (OR) process 31 execut- 
ing on the processor 20 receives packets from the 
crossbar switch 12 interface 15, parses and de- 
codes each packet, places the packet data into an 
intermediate FIFO 32 and performs the necessary 
validity checks of the packet's added header to 
determine if the packet should be received and 
placed at the tail of an enabled queue in packet 
memory 21. Packet reception is established when 
the controller 38 for the crossbar indicates that tnis 
controller 10 is the desired destination port for a 
connection. The OR process 31 indicates its will- 
ingness to take part in a connection by asserting a 
"next" control signal in the interface 15 that the 
crossbar controller 38 samples. When this "next" 



signal is asserted, the crossbar controller may re- 
turn a signal indicating that a connection between a 
source port and this destination port has been 
made on the crossbar 12. The OR process 31 
5 immediately monitors the outbound data received 
at the crossbar interface 15 to delimit a packet. It 
will also deassert the "next" signal and wiil not 
reassert it until it has determined that it is appro- 
priate to establish a new connection. 
w The OR process 31 uses the service class field 

and one specific protocol ciass field value from the 
added header of the outbound packet to place the 
packet on a queue in external packet memory 21. 
The OR process 31 also handles multiple excep- 
ts tion conditions on the crossbar including parity 
errors in the symbols, and packets with bad de- 
limiters. 

The outbound transmit (OT) process 33 ser- 
vices the queues in packet memory 21 that contain 

20 packets to be transmitted to the FDD! line 18 via 
path 19b and device 16. These packets are stored 
in the external packet memory 21 of the controller 
10. When an enabled queue has a packet count 
greater than one, the OT process 33 will begin 

25 processing the packet by initiating packet memory 
requests to move the data from the external mem- 
ory 21 to an internal fifo 34. The OT process 33 
will discard the packet if the time stamp stored with 
the packet has expired, otherwise it wili request the 

30 MAC device 16 to begin a transmission to the FDDI 
line 18. If the OT process 33 is operating in a 
bridge link mode, it wiil decapsulate the packet by 
removing the added header that was stored with 
the packet, prior to transmission on the path 19b to 

35 the FDDI link. If the OT process 33 is attached to a 
switch link, it will keep and update the appropriate 
fields of the added header during transmission to 
the path 19b and FDDI link. 

The packet memory (PM) access process 35 

40 controls the interface 22 to the external packet 
memory 21. The packet memory 21 is used to hold 
the packet data of the multiple inbound and out- 
bound queues and is also used to store the PC 
table, the hash table as described herein, and the 

45 translation database used for lookups. The PM 
process 35 controls access to the external packet 
memory 21 by arbitrating among ail the processes 
requesting access to it. The PM process 35 will 
indicate to the requesting process when a cycle 

so has been granted. The external packet memory is 
accessed via a 56-bit data bus in interface 22. The 
PM process 35 can select up to 1 -million 53-bit 
words over its 20-bit external packet memory ad- 
dress bus in interface 22. Read or write cycles can 

55 be performed every 80-ns to provide 600-Mbps of 
packet memory bandwidth. 

When a process 27, 29, 31, or 33 requests a 
memory cycle via paths 36, 37, 42, or 43, respec- 
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tively, it also indicates to the PM process 35 the 
desired queue that the cycle is associated with by 
driving the queue number. When the PM process 
35 grants, the cycle to the requesting process, it 
uses the queue number provided to simultanegusly 
enable the pointers for the queue to the external 
memory 21. The PM process 35 will drive the 
corresponding tail (write cycle) or head (read cycle) 
pointer onto the external address pins of interface 
22 ; and will perform the necessary pointer main- 
tenance when the cycle completes. The mainten- 
ance includes wrapping, the head or tail pointer 
when the boundary of a queue is reached. The PM 
process 35 will also monitor the queue pointers to 
detect and signal when a queue is approaching an 
overflow condition. . . 

Each queue in the external packet memory 21 
must be defined before the PM process 35 can 
perform an access to it. To define a queue, the 
boundaries for the queue, must be specified by 
programming the external memory block addresses 
that point to the beginning and end of each queue. 
A queue is programmable in size in increments of 
256 56-bit word units (1 792 bytes) which constitute 
a block. In addition to defining the size of the 
queue, the owners of the queue head and tail must 
be specified, to control the direction of packet flow. 
A separate queue enable is provided for each 
queue in order to turn each one on and off in- 
dependently. 

The line card processor (LCP) access process 
40 provides the interface between the processor 25 
and the command and status registers 41 within 
the controller 10. Some of these registers are ac- 
cessed directly by the processor 25 and some are 
accessed indirectly using the LCP process 40. The 
registers 41 that are indirectly accessible are 
shared among the processes 27, 29, 31, 33 f and 35 
of the controller 10 and must be arbitrated for. The 
indirect access interface of the LCP process 40 
provides the mechanisms to select the desired 
indirect register and to trigger the desired read or 
write operation. Since the processor 25 does not 
have direct access to the external packet memory 
21 of the controller 10, it must also use the indirect 
access interface of the LCP process 40 to perform 
read and write cycles to the external memory 21 

In addition to., direct and indirect register 41 
access, the LCP process 40 also maintains the 
traffic event counters and interrupt event registers. 
The LCP process 40 monitors the traffic events 
signaled by the processes 27, 29, 31, 33, and 35 to 
update the corresponding counter. In addition to 
specified counters, the LCP process 40 provides 
two general purpose counters that can be pro- 
grammed to count a particular traffic event. The 
LCP monitors events in the controller 10 and will 
set corresponding interrupts. These interrupts are 



categorized to be associated with inbound traffic, 
outbound traffic, or fatal interrupts. All interrupts, 
except fatal interrupts, are maskable. The presence 
of a non-masked interrupt will result in the asser- 

5 tion of an interrupt signal. 

The operation of the processor 20 in controller 
10 using the processes referred to includes a num- 
ber of functions related to inbound and outbound 
packets. These include filtering, address lookup, 

w address matching, rate limiting, parity checking, 
etc. 

The controller 10 performs input port filtering to 
prevent undesired traffic from entering the net, and 
this, filtering is based on certain addresses con- 

75 tained in a packet. A 48-bit destination address and 
a 48-bit source address in the packet can be 
filtered for a number of addresses, up to the size of 
the translation table in memory 21. Certain ad- 
dresses identified in the protocol as IEEE 802.2 

20 LLC DSAP and IEEE 802.2 LLC SNAP can be 
fiitered by checking against items stored in the 
CAM 23. e.g., 256 entries. 

For address lookup, the controller 10 imple- 
ments an optimized hash algorithm which can per- 

25 form a 48-bit address lookup in at most four re- 
ferences to the external memory 21. The, hash 
function is programmable, and is specified by a 48- 
bit value called the hash function. The lookup in- 
cludes one memory reference to a hash table in 

30 memory 21, followed by at most three references 
to the translation database which contains a 
breadth-first binary tree in which hashed 48-bit 
addresses are stored, as will be explained. 

The controller 10 supports exact matching of 

35 certain 48-bit destination addresses (group or in- 
dividual), certain 48-bit source addresses, 8-bit 
DSAP and 40-bit PID fields, by an interface to the 
external CAM 23. The controller 10 can obtain by 
interface 24 a 1 4-bit associated data field for each 

40 entry stored in the CAM 23. The ability to match 
data fields of varying byte widths is achieved by 
specifying a type code field driven by the controller 
10 during the reception of packets. The bits of the 
type code indicate the size of the field to match in 

45 the CAM array 23 and the type of key data. A key 
can be from one to six bytes in length. When a 48- 
bit destination or 48-bit source address is found as 
an entry in the CAM 23, the controller inhibits the 
hash lookup for this value. 

so The controller 10 provides programmable rate 

limits to throttle the packet rate and limit the trans- 
mit queue length of excess packets that are des- 
tined for switch processing engines. For example, 
these rate limits would be applied to errored pack- 

55 ets, monitor packets, or bridge learning packets 
with an unknown source address to be sent to the 
controller 38 for the switch 12. 
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The controller 10 performs output port filtering 
based on the added header protocol class, and 
performs parity checking of the crossbar data path. 

The controller 10 can be used to attach a 
variety of links to the crossbar 12. The parameters 
that define the operation of controller 10 describe 
the type of link to which the controller 10 is at- 
tached through the MAC interface 19a, 19b, the 
type of inbound and outbound forwarding proce- 
dures it is to perform, and the format of the pack- 
ets it is to handle. 

All externa! links are attached to the controller 
10 through the MAC interface 19a, 19b. Any phys- 
ical channe! can be attached to the controller 10 
through a formatter that adheres tc the specifica- 
tion of the MAC receive and transmit data paths 
19a, 19b. The MAC interface 19a, 19b and the 
controller 10 can operate in either half or full du- 
plex modes. The crossbar interface 14, 15 of the 
controller 10 is a full duplex interface that can 
establish connections and transmit to the crossbar 
12 while simultaneously accepting connections and 
receiving frames from the crossbar. 

In addition to the behavior of the links (i.e., data 
rate and duplex mode), the services that the con- 
troller 10 is to provide for the traffic on the attached 
link 1 1 must be specified. This aspect of link speci- 
fication defines the controller 10 inbound and out- 
bound forwarding/filtering procedures and addition- 
ally defines the format of the frames which the 
controller 10 will process. The controller 10 can 
attach to three types of links, particularly bridge 
links, internal switch links, and relay links. 

A bridge link attaches a network segment 1 1 of 
an extended LAN to the net. That is, referring to 
Figure 3, what may be thought of as a higher level 
network 50 (referred to also as the "GigaNet") 
contains the crossbar switch 12 as one of its ele- 
ments, and the controller 10 is the bridge between 
the FDDI link 1 1 and the net 50. Of course, many 
other LANs such as the FDDI 1 1 or Ethernet LANs, 
etc., would be connected to the net 50 via other 
ports 13 of the crossbar switch 12, or through other 
crossbar switches 12 in the net 50 (using other 
controllers 10 or their equivalent). Within the net 
50. the crossbar device 12 may function as classic 
point-to-point connector, or as set forth in the 
above-mentioned copending applications, may cre- 
ate a ring or bus, or may be create a mixed 
configuration of rings, buses and point-to-point con- 
nectors. The controller 10 will expect to receive 
and transmit frames in FDDI format at the MAC 
interface 19a, 19b, and will perform inbound 
lookups on the destination address, source ad- 
dress, and protocol fields, and translate the packet 
to switch frame format. The controller 10 will per- 
form outbound checks and will perform protocol 
class lookup on the switch frames received from 



the crossbar 12 and translate the frames. back to 
FDDI frame format prior to, transmission to the 
MAC interface 19a, 19b. 

An internal switch link connects switches in the 
5 net 50 together or attaches net end nodes to the 
net 50. Again, the controller 10 can operate in this 
switch link mode. In this mode, the controller 10 
will expect to receive and transmit frames in switch 
. frame format at its MAC interface 19a, 19b. It will 

w perform inbound checks on the switch frame and 
transmit the packet to the crossbar 12. The control- 
ler 10 will perform outbound checks on the switch 
frame received from the crossbar and will transmit 
to the MAC interface 19a, 19b a switch frame. 

75 A relay link uses a full duplex FDDI link to 
attach switches 12 in the net 50 together. The 
controller 10 functioning in this mode will expect to 
receive and transmit frames in a relay frame format 
at the MAC interface 19a, 19b, and here its opera- 

20 tion is similar to switch link attachment and simply 
translates the packet format by adding or stripping 
a local packet header. 

The type of link (bridge link, switch link, or 
relay link) that the controller 10 is attached to at the 

25 MAC interface 19a, 19b is specified by either de- 
coding the frame control field of the received pack- 
et or by a type field in a control register 41 within 
the controller 1 0. 

Once, the link type that the controller 10 is 

30 attached to is specified, the inbound and outbound 
processes 27. 29, 31, and 33 know the format of 
the frames that will be processed and the forward- 
ing procedures it is to perform. 

Referring to Figure 4, the three different frame 

35 formats that may exist at the MAC interface 19a. 
19b are illustrated, including an FDDI frame format 
54, a switch frame format 55 (which encapsulates 
the FDDI frame 54), and a relay frame format 56 
(which encapsulates the switch format 55). 

40 The original FDDI frame format 54 is of the 
form specified by the FDDI protocol, and will not 
be described in detail. Within the FDDI format 54 
are an FDDI header 57, an 802.2 LLC header 58, 
an information field 59 (the payload), and an FDDI 

45 CRC field 60. Various other elements such as 
starting and ending delimiters are present. Within 
the header 57 are 48-bit source and destination 
addresses 61 and 62, and these are used in the 
address lookups as described herein. Other special 

so addresses or identification specified in the protocol 
are contained in the LLC header 58, such as the 
ones mentioned above (DSAP and SNAP). Each of 
these addresses is extracted by the controller 10 
and used for lookups and filtering as described 

55 herein. The FDDI frame format 54 is found at the 
MAC interface 19a, 19b of the controller 10. When 
attached to bridge links, this will be the format of 
the frames received by the IR process 27 and will 
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be the format of frames transmitted by the OT 
process 33. 

The switch frame format 55 of Figure 4 is seen 
to be merely the original FDDI frame 54 to which is 
attached at the beginning an added header 63, also 
referred to as the "gigaheader". This added header 
63 is described in detail below and is ah important 
feature of 'the invention. The switch fram^ format 
55 can appear on both the MAC interface 19a. 19b 
and the crossbar interface 14, 15: At the crossbar 
interface 14, 15, this is the common packet format 
used by the switch piorts 13. It is the format trans- 
mitted by the IT process 29, and it is the format 
received by the OR process 31 . 

The relay frame format 56 of Figure 4 consists 
of a switch frame format 55 (including FDD! format 
54 and added header 63). to which is added a local 
FDDI header 64 and a local FDDI CRC field 65. 
The local FDDI header 64 is of the same specifica- 
tion as the FDDI header 57, arid is defined in the 
FDDI protocol. This type of frame 56° is found at 
the MAC interface 19a, 19b of the controller 10 
when it is attached to a relay link. 

Referring to Figure 4, the added header 63 (or 
"gigaheader") is a thirteen byte (104-bit) field that 
is present in switch frames 55 and relay frames 56. 
It contains information that is generated by the 
inbound receive (IR) process 27 when a packet first 
enters the net 50 and is used to direct the packet 
to its destination within the net 50. This field 63 
defines how the packet is to be serviced within the 
net 50. The packet processing within the net 50 is 
performed by examining the fields of this added 
header 63. Each of the fields in the added header 
63 will now be described, referring also to Figure 5. 

The service class field 65 is a four-bit field that 
specifies the service class for the packet. For the 
receivers of the controller 10, the service class is 
used to determine the queue number in memory 
21 that a packet is to be placed on by doing a 
lookup using certain internal registers 41. On the 
crossbar 12, the service class is also used for 
connection queue servicing. 

The destination switch field 66 is a 12-bit field 
representing the switch number (i.e., local address) 
of the final destination of the packet within the net 
50. When a packet is received from an FDDI link 
11, the IR process 27. performs a lookup operation 
to derive the destination switch value for the pack- 
et. A 12-bit field allows 4096 ports* a single cross- 
bar 12 has only thirty-six ports in one embodiment, 
so the need for a 12-bit address is because a 
number of the crossbars 12 may be within a net 
50, and to allow for expansion to larger crossbars 
and larger nets 50. Certain switch values are re- 
served: 02 h ex and 03 he x indicate that the final des- 
tination is unknown to the IR process 29, and 00 he x 
and 01 h ex indicate that the final destination address 



is to be filtered at this port. Other values derived 
' for this field will direct the packet to its destination 
within the switch. 

The decapsulate bit 67, when set to one, in- 

5 dicates that this packet is to be decapsulated when 
it goes out onto a link that supports both relay and 
bridge frames. The outbound processes 31 and 33 
of the controller 10 will sample this bit td make a 
determination of the transmission format to use. 

w Usually this bit is set to zero: 

' ■ : The destination link 68 is a 7-bit field repre- 
senting the logical link number of the final destina- 
tion of the packet, except when* switch field 66 
equafs OOhex. in which case this field 68 is the 

75 physical link number bf the port that is to receive 
the message: The IR process 2!9 will generate this 
field when constructing the added I* header 63 for 
frames received from the FDDV interface 19a, 19b. 
When enabled, the OR process 31 will' compare 

20 this field to its own logical link number to verify that 
the crossbar connection is established in the cor- 
rect port. Logical link nurhbers 00-to-07 hex are re- 
served for addressing the switch control processors 
x for switches 12 in the net 50. 

25 The protocol class field 69 of Figure 5 is an 8- 

bit field representing the protocol ciass assigned to 
the packet within the net 50. The IR process 29 will 
generate this field 69 when constructing the added 
header 63 for frames received from the FDDI inter- 

30 face 19a, 19b. The OR process 31 will sample this 
field 69 for frames received from the crossbar 12 to 
determine if the packet is enabled to be forwarded 
in the outbound direction. The values F0-to-FD hex 
of this field 69 are reserved for special net-wide 

35 protocols such as inter-switch control messages, 
initialization protocols, implementation of protocol 
trapping, and frames destined for a line control 
processor 25. The values FE and FF hex are used as 
filter protocol values. All other numbers are user- 

40 assignable. 

The source link type field 70 is a four bit field 
representing the type of data link that sourced the 
packet that the current added header 63 is at- 
tached to. This field 70 can be used to specify 

45 additional translation processing during the for- 
warding of the frame. The controller 10 does not 
use this field during packet processing. 

The source switch field 71 is a 1 2-bit field 
representing the switch number of the original 

50 source of this packet in the net 50. The values 00- 
to-03hex are illegal on all data links, and certain 
ones are reserved; all other numbers are user 
assignable. 

The source link field 72 is a 7-bit field repre- 
ss senting the logical link number of the original 
source of this packet in the net 50. The values 00- 
to-07 he x are reserved for use by the switch control 
processor x. 
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The previous hop link field 73 is a 7-bit field 
• representing the physical link number of the pre- 
vious hop link. Thus it represents the link number 
that is currently sending the packet to the crossbar 
12. 

The translation status field 74 is a 4-bit field 
which stores the result of the automatic translation 
performed on the current packet. The status may 
indicate whether the frame is to be forwarded to its 
destination, monitored, crapped, logged, or whether 
the source address of the frame was "new." 

The type of time field 75 of Figure 5 is a 1-bit 
field representing the format for the timestamp in 
the current packet. When this bit is "0" the 
timestamp field represents absolute time, and when 
set to "1" the time stamp field is interpreted as a 
relative time to live. 

The end-of-packet stream field 76 is a 1-bit 
field which, when set to one, indicates that this 
packet is the last packet in a packet stream. Other- 
wise, when set to zero, it indicates the first or a 
middle packet in a packet stream. 

The system port field 77 is a 1-bit field which, 
when set to one, indicates that this packet is being 
sent from a system port (e.g., a switch control 
processor x port). This bit can be used during 
outbound packet processing to restrict certain 
types of packet to originate only from system 
ports. 

The copied frame field 78 is a 1-bit field which, 
when set to one, indicates that a copy frame is 
being sent to the crossbar 12. Copy frames are 
used tc send exception traffic to engines within the 
switch 12. 

The outbound congestion bit 79 is a 1-bit field 
that is used to associate outbound congestion in- 
formation with the added header 12 of the current 
packet. 

The inbound congestion bit 80 is a 1-bit field 
that is used to associate inbound congestion in- 
formation with the added header 12 of the current 
packet. 

The timestamp field 81 is a 16-bit field repre- 
senting a vaiue from 00 to 65535 which is either 
the lowest 16-bits of an absolute time that the 
packet must be destroyed, or the number of time 
units the packet has to live. One time nit is 10- 
milliseconds. The timestamp is generated by the 
inbound receive process 27. 

Reserved field 82 represents 4-bits reserved 
for future specification. These should be sent as 
zeros and otherwise ignored. 

The header check field 83 is an 8-bit CRC 
covering the previous 96-bits of the 104-bit added 
header 63 (the "gigaheader"). 

The frame translation performed within the con- 
troller 10 will now be described. The controller 10 
forwards inbound and outbound frames and per- 



forms the necessary translation to generate the 
desired packet formats 54-56, e.g., adding the 
header 63 if necessary. The forwarding and transla- 
tion procedures that are performed by the pro- 
5 cesses 27, 29, 31, and 33 of the controller 10 are a 
function of the link type that the controller 10 is 
attached to and of course a function of whether the 
packet is inbound or outbound. 

For inbound processing of a packet received 

w where the MAC link type is "bridge" and the MAC 
frame format is FDDI format 54, with the crossbar 
switch 12 frame format being "switch" format 55, 
the actions performed are: 

Perform a lookup of the frame control field of 

75 the received packet. Process the destination ad- 
dress 61 , source address 62, and protocol fields of 
the inbound FDDI frame 54. Determine if the pack- 
et is to' be forwarded/monitored and if there are any 
exception conditions. Generate the service class 

20 field 65, the destination switch field 66, the destina- 
tion link field 68. and the protocol class field 69. 
Add a timestamp field 82 and the remaining header 
63 fields to form a switch frame 55. Queue the 
packet in memory 21. 

25 For inbound processing of a packet received 

where the MAC link type is "bridge" and the MAC 
frame format is "bridge 7 ' format 55, with the cross- 
bar switch 12 frame format being "switch" format 
55, the actions performed are: 

30 Perform a CAM lookup in CAM 23 using the 

protocol class field 69 of the received header 63. 
Check if the packet is to be monitored or trapped. 
Clear the system port bit 77 and check the 
timestamp 82 and checksum 83 for the header 63. 

35 Add the updated header 63 to the frame. 

For inbound processing of a packet received 
where the MAC link type is "relay" format 56 and 
the MAC frame format is "relay" format 56, with 
the crossbar switch 12 frame format being "switch" 

40 format 55, the actions performed are: 

Perform a lookup of the frame control field of 
the received packet. The local header 64 is 
checked and removed from the relay frame 56. 
The procedure then proceeds as specified by the 

45 switch link. 

For outbound processing of a packet received 
where the MAC link type is "bridge," the crossbar 
switch 12 frame format is "switch" format 55, and 
the MAC frame format is FDDI format 54, the 

so actions performed are: 

Check added header 63 checksum 83 for the 
received frame. Check the fields of the header 63, 
including destination switch field 66, destination link 
field 68, and system port bit 77. Perform protocol 

55 class lookup for field 69. Verify that the timestamp 
82 is still good, it not discard the frame. Strip 
header 63 from the packet to realize an FDDI frame 
format 54. 



8 



15 



EP 0 594 199 A1 



16 



For outbound processing of a packet received 
where the MAC link type is "switch," the crossbar 
switch 12 frame format is "switch" format 55, and 
the MAC frame format is "switch" format 55, the 
actions performed are: 

Check the added header 63 checksum 83 for 
the received frame. Check the fields of the header 
63, including destination switch field 66, destination 
link field 68, and system port bit 77. Perform pro- 
tocol class lookup for field 69. Verify that the 
timestamp 82 is still good, if not discard the frame. 

For outbound processing of a packet received 
where the MAC link type is "relay," the crossbar 
switch 12 frame format is "switch" format 54, and 
the MAC frame format is "relay" format 56, the 
actions performed are: 

The local header 64 is added to the switch 
frame 55. The procedures then proceed just as 
specified by the switch link, i.e., the frame is treat- 
ed as if it were an FDDI frame 54. 

During the frame forwarding procedure, the 
controller 10 makes use of several lookup mecha- 
nisms to process the fields of the frame. These 
include the CAM lookup, the hash lookup, a class 
specifier lookup, and a protocol class lookup. 

The CAM lockup consists of applying an ad- 
dress or value to the CAM 23, and getting back an 
indication of whether or not a match is found in the 
CAM 23. The CAM 23 is of limited size due to the 
cost (number of transistors) inherent in constructing 
a CAM. In an example embodiment,- the CAM 23 
holds 256 entries. Thus, there can be 256 values 
(addresses, etc.) that are searched for in a CAM 
lockup. The IR process 27 uses the CAM lookup to 
perform matches using the various fields of the 
received frame 54, 55, or, 56 as keys. Variable 
width key matching is provided, up to six bytes 
wide, and this is specified by the IB process 27 
during the parse of the frame, and a match result 
return an associated data value that will be used by 
the IR process 27. This CAM match can be used to 
find the frame control, destination address, source 
address, DSAP and SNAP fields of bridge and 
relay link packets. It can also be used to find the 
protocol class field 69 of the added header 63 of 
the switch link packets 55. 

A protocol class lookup is performed by the 
OR process 31 to , determine if the protocol class 
field 69 in the added header 63 for a frame re- 
ceived from the crossbar switch 12 via line 15 
indicates that this protocol class is enabled to be 
forwarded outbound. 

A hash lookup is used by the IR process 27 (in 
conjunction with a CAM lookup) to process the 48- 
bit destination and source fields 61 and 62 of 
received frames. The 48-bit addresses 61 and 62 
may be globally unique addresses, i.e., each sta- 
tion may have an address that is unique to that 



station. The address range covers 2 40 or approxi- 
mately 3x1 0 14 (300 trillion) unique values, so an 
address may never need to be duplicated in any 
foreseeable systems. Within an extended network, 
5 however, the number of unique addresses needed 
is usually only a few thousands or at most tens of 
thousands. Therefore, a table of all of the ad- 
dresses being used in a network at a given con- 
figuration only contains, for example, a maximum 
w of 64K or 2 16 , entries, which would use merely a 
16-bit address. For this reason, the 48-bit globally- 
unique address 61 or 62 is hashed, producing a 
1 6-bit. Jocally-unique address. An incoming address 
field 85 as seen in Figure 8 is subjected to a hash 
15 function 86 to produce another 48-bit value 87, 
then a 16-bit part 88 of this 48-bit value 87 (the 
least significant 16-bit field) is used to index into a 
64K-entry table in memory 21, this being referred 
to as the hash table 89. Each word of the memory 
20 21 is 56-bits wide, so a word can contain three of 
. thesje hash entries, in three "hash buckets" 90 for 
each word 91, as seen in Figure 6. When a hashed 
address 87 is generated during the processing of 
packets, the 16-bit index 88 is used to select one 
25 of the 22K words 91 in the hash table 89 and one 
of the three hash buckets 90 at this index. Each 
/ bucket 90 is an 18-bit field containing a 3-bit value 
92 indicating the bucket size, one to seven entries, 
and a 15-bit field 93 acting as a translation table 
30 pointer. The 16-bit input 88 indexes to a word 91 
depending upon its lower order bits and one-of- 
three buckets 90 in a word 91 depending upon its 
higher ordered bits, indexed for one-of-22K selec- 
tion of the word 91 in the table. The translation 
35 table pointer 93 returned by the hash bucket 90 in 
the hash table 89 is used to select a breadth-first 
balanced binary tree as illustrated in Figure 7. The 
trees are stored in a translation table 94 in memory 
21, and each tree has between one and seven 
40 entries, as indicated by the size field 92. The 
binary tree cannot cress a block boundary in the 
translation table 94 in memory 21 . The ordering of 
entries 96 in a breadth-first balanced binary tree for 
various table sizes is illustrated in Figure 7. Note 
45 that there are from one to seven entries (each entry 
a 48-bit address) in a tree, ordered such that 

A < B < C < D < E < F < G 

50 Each entry 96 contains 32-bit hash remainder field 
97 that is left after using the low-order 16-bit value 
88 from the hashed 48-bit address to store an 
address. To traverse the tree, the IR process 
fetches the first entry 96, and the 32-bit value 97 in 

55 this entry is compared to the upper 32-bit value 97 
of the incoming hashed address 87 (from the pack- 
et being evaluated). If the two values match, the 
remaining fields of the entry 96 in the translation 
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table 94 provide the IR process 27 with the in- 
formation it needs for processing that field of the 
packet. Otherwise, the IR process will continue 
traversing the binary tree by selecting the next 
entry as a function of whether the has remainder is s 
less than or greater than the current entry 96. The 
hash remainders 97 are compared, and if the in- 
coming value 97 is less than the stored value 97 
the left branch 98 is picked, while if greater than 
the right branch 99 is picked. The tree traversal will to 
continue . until a match is found until a match is 
found or the tree is exhausted. For each hash 
lookup and tree traversal, up to four memory re- 
quests to external memory 21 will be made by the 
IR process 27. One memory request is required to /5 
obtain the hash bucket 90, and then up to three 
more memory requests are needed to traverse the 
binary tree of Figure 7 (e.g.. D - B - C of the 7- 
entry tree). The arbiter of the PM process 35 
provides priority service for memory requests asso- 20 
ciated with this search process in the IR process 
27, and will allocate up to every other memory 
cycle for the IR lookup during packet reception. 

The address lookup procedure used by the IR 
process 27, as just described, is also illustrated in 25 
Figure 8. The 48-bit destination address 61 from an 
incoming packet is simultaneously subjected to the 
hash function 86 and a CAM match; if a CAM 
match is found in CAM 23, an index into the 
translation table 94 is immediately generated as 30 
indicated by the line 100. If not, the 16-bit field 88 
of the hashed address 87 is used to index into the 
hash table 89 to select a hash bucket 90, and this 
selected hash bucket contains a pointer 93 into the 
translation table 94. The binary tree found in the 35 
translation table 94 will have a size indicated by the 
3-bit field 92. Each entry 96 in the tree has a 32-bit 
remainder field 97 which is compared at compara- 
tor 101 with the 32-bit remainder in the hashed 
address 87 to either find a match or progress 40 
through the tree from a maximum of three fetches. 

When the hash function 86 is implemented to 
build the hash table 89 and translation table 94, as 
upon initialization of a network and whenever the 
network is reconfigured by adding stations or brid- 45 
ges, there is a small but finite probability that more 
than seven addresses will hash to the same bucket. 
If this occurs, the address producing the collision in 
trying to load the table will be stored in the CAM 
23. Perhaps thirty-two or so of these collision ad- so 
dresses can be stored in the 256-entry CAM 23 
without degrading its other functions. Thus, since 
every address is compared with the CAM entries 
anyway (to filter for multicast messages, SNAP 
filtering, etc.), and this compare in the CAM is done 55 
in parallel with the hash function, the CAM com- 
pare is without cost in time or new circuitry. Were 
this outlet not available, the occurrence of more 



than seven entries for a hash .bucket would result in 
the necessity to chose a new hash function and 
recalculate all the addresses to rebuild the hash 
table 89 and the translation table 94; this would 
require a period of down time where the controller 
10 could not respond to incoming traffic since the 
hash and translation tables would be unavailable, 
and processor time would be monopolized by the 
rebuild. The 48-bit source and destination address- 
ees are often not randomly selected, but instead are 
assigned in blocks or patterns and otherwise in- 
crease the likelihood of hashing to the same buck- 
et. Alternatively, one way of avoiding the con- 
sequences of having to rebuild the hash table 89 is 
to generate two hash tables in memory 21 using 
two different hash functions, so if the one being 
used generates a collision (more than seven in a 
hash bucket), processing switches to the alternate 
hash table where a collision is very unlikely; this 
requires extra size for memory 21, however. The 
availability of the CAM lookup for instances where 
there are a few collisions in loading the hash buck- 
ets has the advantage of not imposing any signifi- 
cant burden in time or in memory usage. 

While the invention has been described with 
reference to a specific embodiment, the description 
is not meant to be construed in a limiting sense. 
Various modifications of the disclosed embodiment, 
as well as other embodiments of the invention, will 
be apparent to persons skilled in the art upon 
reference to this description. It is therefore con- 
templated that any such modifications or embodi- 
ments which fall within the true scope of this inven- 
tion. 

Claims . 

1. A packet data communications network com- 
prising: 

a first network segment having a plurality 
of stations, one of said stations sending a 
message packet onto said first network seg- 
ment of a first format: said first format includ- 
ing a first header and a data field with network 
destination address in said communications 
network; 

a first network transfer device having an 
input connected to said first network segment 
to receive said message packet and having an 
output; the first network transfer device apply- 
ing a second header to said message packet, 
said second header including a switching ad- 
dress translated from said destination address 
and including local status information; 

a switching device having a plurality of 
ports, a first of said ports being connected to 
said output of said first network transfer de- 
vice; the switching device receiving said mes- 
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sage packet with said second header and 
sending said message packet with said second 
header to a second port as selected by said 
switching address, and in response to Said 
local status information; 

a second network transfer device having 
an input connected to said second of said 
ports of r said switching device and having an 
output connected to a second ' c network seg- 
ment, the second network transfer device re- 
ceiving said message packet via said switching 
device to forward to said second network seg- 
ment; the second network transfer device ; re- 
moving said second header from said mes- 
sage packet. 

2. A network according to claim * 1 wherein said 
network has' a plurality of links, and each of 
said links is assigned a link number/ and said 
second header includes a link number for a 
source of said message packet and a link 
number for a destination of said message 
packet, and wherein said destination address 
contains N bits, and said switch address con- 
tains M bits, where N and M are integers and 
N » M. 

3. A network according to claim 2 wherein said 
packet includes a network source address of N 
bits, and said added header contains a source 
switch address of M bits translated from said 
network source address, wherein said switch- 
ing device is a crossbar switch wherein said 
first network segment is a serial FDDI link, and 
said ports are parallel ports, further wherein 
said added header contains a service class 
field, and said switching device processes said 
packet in response to said service class field. 

4. A network according to claim 3 wherein said 
added header contains a protocol class field, 
and said switching device processes said 
packet in response to said protocol class field, 
wherein said added header contains a status 
field indicating local congestion, and said 
switching device processes said packet in re- 
sponse to said status field. 

5. A method of operating a packet data commu- 
nications network, the network including a first 
network segment having a plurality of stations 
and a second network segment having a plu- 
rality of stations, and including a switching 
device interconnecting said first and second 
segments, comprising the steps of: 

sending from one of said stations of said 
first network segment a message packet of a 
first format onto said first network segment; 



said first format including a first header and a 
data field with a network destination address in 
said communications network; 

'receiving said message packet at a first 

5 network transfer device having an input con- 

nected to said first network segment; the first 
network' transfer device applying a second 
header to said message packet, said second 
header including a switching address trans- 

10 "■• -lated from said destination address and includ- 
ing local status information; 

■receiving at said switching device said 
message packet with said second header and 
sending said message' packet with said second 

75 header to a port of said switching device as 

selecterd by said switching address, and in 
response to said local status information; 
- " receiving said message packet at said 

second network transfer device via said switch- 

20 ing device and forwarding said message pack- 

et to said second network segment; the sec- 
ond network transfer device removing said 
second header from said message packet. 

25 6. A method according to claim 5 wherein said 
network has a plurality of links, and each of 
said links is assigned a link number, and in- t 
serting in said second header a link number for & 
a source of said message packet- and a link . . ■ 

30 number for - a destination of said message 

packet, wherein said destination address con- 
tains N bits, and said switch address contains *• 
M bits, where N and M are integers and N >> 
M. 

35 

7. A method according to claim 6 wherein said 
packet includes a network source address of N 
bits, and inserting in said added header a 
source switch address of M bits translated 

40 from said network source address, wherein 

said switching device is a crossbar switch, 

8. A method according to claim 7 including the 
steps of: transmitting on said first network seg- 

45 ment by the serial FDDI method, and ports 

between said switching device and said trans- 
fer devices are parallel ports; and inserting in 
said added header a service class field, and 
said switching device processes said packet in 

50 response to said service class field. 

9. A method according to claim 8 including in- 
serting in said added header a protocol class 
field, and said switching device processes said 

55 packet in response to said protocol class field. 

10. A method according to claim 9 including in- 
serting in said added header a status field 
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indicating local congestion, and said switching 
device processes said packet in response to 
said status field. 
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